{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# 数据分析基础\n",
        "\n",
        "## 数据获取和保存\n",
        "\n利用R语言中原始的`read.table()`函数读取测试文件："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "students <- read.table('../Data/students-read.txt', header=T)\n",
        "print(students)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  ID   Name Gender Score\n",
            "1  1    Leo   Male    90\n",
            "2  2    Tom   Male    80\n",
            "3  3 Olivia Female    89\n",
            "4  4   John   Male    77\n",
            "5  5   Jacy Female    83\n"
          ]
        }
      ],
      "execution_count": 1,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "利用R语言中原始的`write.table()`函数写入测试文件："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "colleagues <- data.frame(\n",
        "    names = c('Leo', 'Tom', 'Jacy'),\n",
        "    gender = c('Male', 'Male', 'Female'),\n",
        "    age = c(26, 24, 25)\n",
        ")\n",
        "write.table(colleagues, '../Data/colleagues-write.txt', quote=F, sep='\\t')"
      ],
      "outputs": [],
      "execution_count": 2,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "利用readr包读取测试文件："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "library(readr)\n",
        "students <- read_tsv('../Data/students-read.txt', col_names=T, col_types=cols('c', 'c', 'c', 'd'))\n",
        "print(students)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 5 x 4\n",
            "     ID   Name Gender Score\n",
            "  <chr>  <chr>  <chr> <dbl>\n",
            "1     1    Leo   Male    90\n",
            "2     2    Tom   Male    80\n",
            "3     3 Olivia Female    89\n",
            "4     4   John   Male    77\n",
            "5     5   Jacy Female    83\n"
          ]
        }
      ],
      "execution_count": 3,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "利用readr包写入测试文件："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "write_tsv(colleagues, '../Data/colleagues-write.txt', col_names=T)"
      ],
      "outputs": [],
      "execution_count": 4,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 数据变换\n",
        "\ndplyr除了包含了很多实用的数据变换函数外，还为我们带来了新的数据类型和语法。dplyr中提供了一种名为tbl的数据类型，tbl类型数据相比R中的数据框，更方便用户查看。`tbl_df()`函数可以将一个数据框转换成为一个tbl类型数据，tbl类型数据可以根据显示区的大小自动显示合适的行数。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "library(dplyr)\n",
        "library(tidyr)\n",
        "\n",
        "options(max.print=30)\n",
        "options(width = 100)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n",
            "Attaching package: ‘dplyr’\n",
            "\n",
            "The following objects are masked from ‘package:stats’:\n",
            "\n",
            "    filter, lag\n",
            "\n",
            "The following objects are masked from ‘package:base’:\n",
            "\n",
            "    intersect, setdiff, setequal, union\n",
            "\n"
          ]
        }
      ],
      "execution_count": 5,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(iris)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Length Sepal.Width Petal.Length Petal.Width    Species\n",
            "1            5.1         3.5          1.4         0.2     setosa\n",
            "2            4.9         3.0          1.4         0.2     setosa\n",
            "3            4.7         3.2          1.3         0.2     setosa\n",
            "4            4.6         3.1          1.5         0.2     setosa\n",
            "5            5.0         3.6          1.4         0.2     setosa\n",
            "6            5.4         3.9          1.7         0.4     setosa\n",
            " [ reached getOption(\"max.print\") -- omitted 144 rows ]\n"
          ]
        }
      ],
      "execution_count": 6,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_tb <- tbl_df(iris)\n",
        "print(iris_tb)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 150 x 5\n",
            "   Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
            "          <dbl>       <dbl>        <dbl>       <dbl>  <fctr>\n",
            " 1          5.1         3.5          1.4         0.2  setosa\n",
            " 2          4.9         3.0          1.4         0.2  setosa\n",
            " 3          4.7         3.2          1.3         0.2  setosa\n",
            " 4          4.6         3.1          1.5         0.2  setosa\n",
            " 5          5.0         3.6          1.4         0.2  setosa\n",
            " 6          5.4         3.9          1.7         0.4  setosa\n",
            " 7          4.6         3.4          1.4         0.3  setosa\n",
            " 8          5.0         3.4          1.5         0.2  setosa\n",
            " 9          4.4         2.9          1.4         0.2  setosa\n",
            "10          4.9         3.1          1.5         0.1  setosa\n",
            "# ... with 140 more rows\n"
          ]
        }
      ],
      "execution_count": 7,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "dplyr还提供了一种描述数据的函数glimpse()，功能类似summary()和str()函数。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "summary(iris)"
      ],
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  Sepal.Length    Sepal.Width     Petal.Length    Petal.Width          Species  \n",
              " Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100   setosa    :50  \n",
              " 1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300   versicolor:50  \n",
              " Median :5.800   Median :3.000   Median :4.350   Median :1.300   virginica :50  \n",
              " Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199                  \n",
              " 3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800                  \n",
              " Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500                  "
            ]
          },
          "metadata": {}
        }
      ],
      "execution_count": 8,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "str(iris)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "'data.frame':\t150 obs. of  5 variables:\n",
            " $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...\n",
            " $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...\n",
            " $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...\n",
            " $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...\n",
            " $ Species     : Factor w/ 3 levels \"setosa\",\"versicolor\",..: 1 1 1 1 1 1 1 1 1 1 ...\n"
          ]
        }
      ],
      "execution_count": 9,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "glimpse(iris)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Observations: 150\n",
            "Variables: 5\n",
            "$ Sepal.Length <dbl> 5.1, 4.9, 4.7, 4.6, 5.0, 5.4, 4.6, 5.0, 4.4, 4.9, 5.4, 4.8, 4.8, 4.3, 5.8,...\n",
            "$ Sepal.Width  <dbl> 3.5, 3.0, 3.2, 3.1, 3.6, 3.9, 3.4, 3.4, 2.9, 3.1, 3.7, 3.4, 3.0, 3.0, 4.0,...\n",
            "$ Petal.Length <dbl> 1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5, 1.5, 1.6, 1.4, 1.1, 1.2,...\n",
            "$ Petal.Width  <dbl> 0.2, 0.2, 0.2, 0.2, 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 0.2,...\n",
            "$ Species      <fctr> setosa, setosa, setosa, setosa, setosa, setosa, setosa, setosa, setosa, s...\n"
          ]
        }
      ],
      "execution_count": 10,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 选取符合条件的的子集数据\n",
        "\n`filter(.data, ...)`，其中.data 为待处理的数据；...为选取子集的条件表达式。例如选取iris数据集中萼片长度大于7的数据："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris_filtered <- filter(iris, Sepal.Length > 7)\n",
        "print(iris_filtered)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "   Sepal.Length Sepal.Width Petal.Length Petal.Width   Species\n",
            "1           7.1         3.0          5.9         2.1 virginica\n",
            "2           7.6         3.0          6.6         2.1 virginica\n",
            "3           7.3         2.9          6.3         1.8 virginica\n",
            "4           7.2         3.6          6.1         2.5 virginica\n",
            "5           7.7         3.8          6.7         2.2 virginica\n",
            "6           7.7         2.6          6.9         2.3 virginica\n",
            " [ reached getOption(\"max.print\") -- omitted 6 rows ]\n"
          ]
        }
      ],
      "execution_count": 11,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "`filter(.data, ...)`支持的过滤函数有：\n",
        "\n",
        "- `==, >, >= etc`\n",
        "- `&, |, !, xor()`\n",
        "- `is.na()`\n",
        "- `between(), near()`"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris_filtered <- filter(iris, Sepal.Length > 7 & Sepal.Width >= 3)\n",
        "print(iris_filtered)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Sepal.Length Sepal.Width Petal.Length Petal.Width   Species\n",
            "1          7.1         3.0          5.9         2.1 virginica\n",
            "2          7.6         3.0          6.6         2.1 virginica\n",
            "3          7.2         3.6          6.1         2.5 virginica\n",
            "4          7.7         3.8          6.7         2.2 virginica\n",
            "5          7.2         3.2          6.0         1.8 virginica\n",
            "6          7.2         3.0          5.8         1.6 virginica\n",
            " [ reached getOption(\"max.print\") -- omitted 2 rows ]\n"
          ]
        }
      ],
      "execution_count": 12,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 删除重复数据\n",
        "\n`distinct(.data, ...)`，其中.data为待处理的数据；...为可选的判断重复的数据列，如果省略该参数则根据全部数据列判断数据是否重复，也可以利用计算后的值作为判断是否重复的条件。例如："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "dim(iris)"
      ],
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<ol class=list-inline>\n",
              "\t<li>150</li>\n",
              "\t<li>5</li>\n",
              "</ol>\n"
            ],
            "text/markdown": [
              "1. 150\n",
              "2. 5\n",
              "\n\n"
            ],
            "text/latex": [
              "\\begin{enumerate*}\n",
              "\\item 150\n",
              "\\item 5\n",
              "\\end{enumerate*}\n"
            ],
            "text/plain": [
              "[1] 150   5"
            ]
          },
          "metadata": {}
        }
      ],
      "execution_count": 13,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_distinct <- distinct(iris)\n",
        "print(iris_distinct)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Length Sepal.Width Petal.Length Petal.Width    Species\n",
            "1            5.1         3.5          1.4         0.2     setosa\n",
            "2            4.9         3.0          1.4         0.2     setosa\n",
            "3            4.7         3.2          1.3         0.2     setosa\n",
            "4            4.6         3.1          1.5         0.2     setosa\n",
            "5            5.0         3.6          1.4         0.2     setosa\n",
            "6            5.4         3.9          1.7         0.4     setosa\n",
            " [ reached getOption(\"max.print\") -- omitted 143 rows ]\n"
          ]
        }
      ],
      "execution_count": 14,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_distinct <- distinct(iris, Species)\n",
        "print(iris_distinct)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "     Species\n",
            "1     setosa\n",
            "2 versicolor\n",
            "3  virginica\n"
          ]
        }
      ],
      "execution_count": 15,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_distinct <- distinct(iris, Sepal.Diff=abs(Sepal.Length-Sepal.Width))\n",
        "print(head(iris_distinct))"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Sepal.Diff\n",
            "1        1.6\n",
            "2        1.9\n",
            "3        1.5\n",
            "4        1.5\n",
            "5        1.4\n",
            "6        1.5\n"
          ]
        }
      ],
      "execution_count": 16,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 数据采样\n",
        "\n",
        "```\n",
        "sample_n(tbl, size, replace = FALSE, weight = NULL, .env = parent.frame())\n",
        "sample_frac(tbl, size = 1, replace = FALSE, weight = NULL, .env = parent.frame())\n",
        "```\n",
        "\n其中tbl为待处理的数据；size对于sample_n为采样的个数，对于sample_frac为采样的比例；replace表示是否又放回的抽样；weight为采样的权重。例如："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "# 采样结果每次可能不一样\n",
        "iris_sample <- sample_n(iris, 3)\n",
        "print(iris_sample)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Length Sepal.Width Petal.Length Petal.Width   Species\n",
            "24           5.1         3.3          1.7         0.5    setosa\n",
            "148          6.5         3.0          5.2         2.0 virginica\n",
            "11           5.4         3.7          1.5         0.2    setosa\n"
          ]
        }
      ],
      "execution_count": 17,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# 采样结果每次可能不一样\n",
        "iris_sample <- sample_frac(iris, 0.05, replace=T)\n",
        "print(iris_sample)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "     Sepal.Length Sepal.Width Petal.Length Petal.Width    Species\n",
            "90            5.5         2.5          4.0         1.3 versicolor\n",
            "95            5.6         2.7          4.2         1.3 versicolor\n",
            "10            4.9         3.1          1.5         0.1     setosa\n",
            "113           6.8         3.0          5.5         2.1  virginica\n",
            "10.1          4.9         3.1          1.5         0.1     setosa\n",
            "141           6.7         3.1          5.6         2.4  virginica\n",
            " [ reached getOption(\"max.print\") -- omitted 2 rows ]\n"
          ]
        }
      ],
      "execution_count": 18,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 选择指定行\n",
        "\n",
        "```\n",
        "slice(.data, ...)\n",
        "top_n(x, n, wt)\n",
        "```\n",
        "\nslice可以选择指定的若干行，其中.data为待处理的数据；...为指定选取的行。top_n为选取数据的前n行，其中x为待处理的数据；n为选取的数据行数，如果数据是分组的（即排序值出现重复），则可能返回超过n行；wt为可选的排序变量，如果不传入该参数，则会使用数据中最后一个变量进行排序。例如："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris_slice <- slice(iris, c(1, 2, 4))\n",
        "print(iris_slice)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 5\n",
            "  Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
            "         <dbl>       <dbl>        <dbl>       <dbl>  <fctr>\n",
            "1          5.1         3.5          1.4         0.2  setosa\n",
            "2          4.9         3.0          1.4         0.2  setosa\n",
            "3          4.6         3.1          1.5         0.2  setosa\n"
          ]
        }
      ],
      "execution_count": 19,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_slice <- slice(iris, 1:3)\n",
        "print(iris_slice)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 5\n",
            "  Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
            "         <dbl>       <dbl>        <dbl>       <dbl>  <fctr>\n",
            "1          5.1         3.5          1.4         0.2  setosa\n",
            "2          4.9         3.0          1.4         0.2  setosa\n",
            "3          4.7         3.2          1.3         0.2  setosa\n"
          ]
        }
      ],
      "execution_count": 20,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_top_n <- top_n(iris, 3)\n",
        "print(iris_top_n)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Selecting by Species\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "   Sepal.Length Sepal.Width Petal.Length Petal.Width   Species\n",
            "1           6.3         3.3          6.0         2.5 virginica\n",
            "2           5.8         2.7          5.1         1.9 virginica\n",
            "3           7.1         3.0          5.9         2.1 virginica\n",
            "4           6.3         2.9          5.6         1.8 virginica\n",
            "5           6.5         3.0          5.8         2.2 virginica\n",
            "6           7.6         3.0          6.6         2.1 virginica\n",
            " [ reached getOption(\"max.print\") -- omitted 44 rows ]\n"
          ]
        }
      ],
      "execution_count": 21,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_top_n <- top_n(iris, 3, Sepal.Length)\n",
        "print(iris_top_n)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Sepal.Length Sepal.Width Petal.Length Petal.Width   Species\n",
            "1          7.7         3.8          6.7         2.2 virginica\n",
            "2          7.7         2.6          6.9         2.3 virginica\n",
            "3          7.7         2.8          6.7         2.0 virginica\n",
            "4          7.9         3.8          6.4         2.0 virginica\n",
            "5          7.7         3.0          6.1         2.3 virginica\n"
          ]
        }
      ],
      "execution_count": 22,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 选取指定列\n",
        "\n",
        "```\n",
        "select(.data, ...)\n",
        "```\n",
        "\n",
        "其中.data为待处理数据；...表示选取的列名或符合条件的列，可用的函数如下：\n",
        "\n",
        "- `starts_with(x, ignore.case = TRUE)` 选取起始文本符合要求的列\n",
        "- `ends_with(x, ignore.case = TRUE)` 选取末尾文本符合要求的列\n",
        "- `contains(x, ignore.case = TRUE)` 选取包含指定文本的列\n",
        "- `matches(x, ignore.case = TRUE)` 选取符合表达式x的列\n",
        "- `num_range(\"x\", 1:5, width = 2)` 选取x01至x05列\n",
        "- `one_of(\"x\", \"y\", \"z\")` 选取名称在指定的字符向量中的列\n",
        "- `everything()` 选择所有列"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris_selected <- select(iris, Sepal.Length, Species)\n",
        "print(iris_selected)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Length    Species\n",
            "1            5.1     setosa\n",
            "2            4.9     setosa\n",
            "3            4.7     setosa\n",
            "4            4.6     setosa\n",
            "5            5.0     setosa\n",
            "6            5.4     setosa\n",
            "7            4.6     setosa\n",
            "8            5.0     setosa\n",
            "9            4.4     setosa\n",
            "10           4.9     setosa\n",
            "11           5.4     setosa\n",
            "12           4.8     setosa\n",
            "13           4.8     setosa\n",
            "14           4.3     setosa\n",
            "15           5.8     setosa\n",
            " [ reached getOption(\"max.print\") -- omitted 135 rows ]\n"
          ]
        }
      ],
      "execution_count": 23,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_selected <- select(iris, starts_with('Petal'))\n",
        "print(iris_selected)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Petal.Length Petal.Width\n",
            "1            1.4         0.2\n",
            "2            1.4         0.2\n",
            "3            1.3         0.2\n",
            "4            1.5         0.2\n",
            "5            1.4         0.2\n",
            "6            1.7         0.4\n",
            "7            1.4         0.3\n",
            "8            1.5         0.2\n",
            "9            1.4         0.2\n",
            "10           1.5         0.1\n",
            "11           1.5         0.2\n",
            "12           1.6         0.2\n",
            "13           1.4         0.1\n",
            "14           1.1         0.1\n",
            "15           1.2         0.2\n",
            " [ reached getOption(\"max.print\") -- omitted 135 rows ]\n"
          ]
        }
      ],
      "execution_count": 24,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_selected <- select(iris, ends_with('Width'))\n",
        "print(iris_selected)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Width Petal.Width\n",
            "1           3.5         0.2\n",
            "2           3.0         0.2\n",
            "3           3.2         0.2\n",
            "4           3.1         0.2\n",
            "5           3.6         0.2\n",
            "6           3.9         0.4\n",
            "7           3.4         0.3\n",
            "8           3.4         0.2\n",
            "9           2.9         0.2\n",
            "10          3.1         0.1\n",
            "11          3.7         0.2\n",
            "12          3.4         0.2\n",
            "13          3.0         0.1\n",
            "14          3.0         0.1\n",
            "15          4.0         0.2\n",
            " [ reached getOption(\"max.print\") -- omitted 135 rows ]\n"
          ]
        }
      ],
      "execution_count": 25,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_selected <- select(iris, contains('etal'))\n",
        "print(iris_selected)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Petal.Length Petal.Width\n",
            "1            1.4         0.2\n",
            "2            1.4         0.2\n",
            "3            1.3         0.2\n",
            "4            1.5         0.2\n",
            "5            1.4         0.2\n",
            "6            1.7         0.4\n",
            "7            1.4         0.3\n",
            "8            1.5         0.2\n",
            "9            1.4         0.2\n",
            "10           1.5         0.1\n",
            "11           1.5         0.2\n",
            "12           1.6         0.2\n",
            "13           1.4         0.1\n",
            "14           1.1         0.1\n",
            "15           1.2         0.2\n",
            " [ reached getOption(\"max.print\") -- omitted 135 rows ]\n"
          ]
        }
      ],
      "execution_count": 26,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_selected <- select(iris, matches('.t.'))\n",
        "print(iris_selected)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Length Sepal.Width Petal.Length Petal.Width\n",
            "1            5.1         3.5          1.4         0.2\n",
            "2            4.9         3.0          1.4         0.2\n",
            "3            4.7         3.2          1.3         0.2\n",
            "4            4.6         3.1          1.5         0.2\n",
            "5            5.0         3.6          1.4         0.2\n",
            "6            5.4         3.9          1.7         0.4\n",
            "7            4.6         3.4          1.4         0.3\n",
            " [ reached getOption(\"max.print\") -- omitted 143 rows ]\n"
          ]
        }
      ],
      "execution_count": 27,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_selected <- select(iris, one_of(c('Petal.Length', 'Petal.Width')))\n",
        "print(iris_selected)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Petal.Length Petal.Width\n",
            "1            1.4         0.2\n",
            "2            1.4         0.2\n",
            "3            1.3         0.2\n",
            "4            1.5         0.2\n",
            "5            1.4         0.2\n",
            "6            1.7         0.4\n",
            "7            1.4         0.3\n",
            "8            1.5         0.2\n",
            "9            1.4         0.2\n",
            "10           1.5         0.1\n",
            "11           1.5         0.2\n",
            "12           1.6         0.2\n",
            "13           1.4         0.1\n",
            "14           1.1         0.1\n",
            "15           1.2         0.2\n",
            " [ reached getOption(\"max.print\") -- omitted 135 rows ]\n"
          ]
        }
      ],
      "execution_count": 28,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_selected <- select(iris, -starts_with('Petal'))\n",
        "print(iris_selected)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Length Sepal.Width    Species\n",
            "1            5.1         3.5     setosa\n",
            "2            4.9         3.0     setosa\n",
            "3            4.7         3.2     setosa\n",
            "4            4.6         3.1     setosa\n",
            "5            5.0         3.6     setosa\n",
            "6            5.4         3.9     setosa\n",
            "7            4.6         3.4     setosa\n",
            "8            5.0         3.4     setosa\n",
            "9            4.4         2.9     setosa\n",
            "10           4.9         3.1     setosa\n",
            " [ reached getOption(\"max.print\") -- omitted 140 rows ]\n"
          ]
        }
      ],
      "execution_count": 29,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 数据集概括\n",
        "\n",
        "```\n",
        "summarise(.data, ...)\n",
        "summarise_all(tbl, funs, ...)\n",
        "summarise_if(.tbl, .predicate, .funs, ...)\n",
        "summarise_at(.tbl, .vars, .funs, ..., .cols = NULL)\n",
        "```\n",
        "\nsummarise可以将数据集进行按照指定的函数和数据列进行汇总概括，summarise_each可以对数据集中的所有列应用相同的函数进行汇总概括，summarise_if可以根据条件对相应的列进行汇总概括，summarise_at可以对指定的列进行汇总概括。例如："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris_summarised <- summarise(iris, mean(Sepal.Width))\n",
        "print(iris_summarised)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  mean(Sepal.Width)\n",
            "1          3.057333\n"
          ]
        }
      ],
      "execution_count": 30,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_summarised <- summarise_all(iris, funs(mean))\n",
        "print(iris_summarised)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Warning message in mean.default(Species):\n",
            "“argument is not numeric or logical: returning NA”"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
            "1     5.843333    3.057333        3.758    1.199333      NA\n"
          ]
        }
      ],
      "execution_count": 31,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_summarised <- summarise_if(iris, is.numeric, funs(mean))\n",
        "print(iris_summarised)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Sepal.Length Sepal.Width Petal.Length Petal.Width\n",
            "1     5.843333    3.057333        3.758    1.199333\n"
          ]
        }
      ],
      "execution_count": 32,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_summarised <- summarise_at(iris, c('Sepal.Length', 'Sepal.Width'), funs(mean))\n",
        "print(iris_summarised)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Sepal.Length Sepal.Width\n",
            "1     5.843333    3.057333\n"
          ]
        }
      ],
      "execution_count": 33,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "除了利用summarise()和summarise_each()两个函数进行汇总概括以外，dplyr还提供了系列方便的概括方法，如下：\n",
        "\n",
        "- `count(x, ..., wt = NULL, sort = FALSE)` 计算行数（可带权重）\n",
        "- `nth(x, n, order_by = NULL, default = default_missing(x))` 向量的第n个值\n",
        "- `first(x, order_by = NULL, default = default_missing(x))` 向量的第1个值\n",
        "- `last(x, order_by = NULL, default = default_missing(x))` 向量的最后1个值\n",
        "- `n()` 向量中元素的个数（不可直接调用）\n",
        "- `n_distinct(x)` 向量中的不同元素的个数"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris_count <- count(iris, Species)\n",
        "print(iris_count)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 2\n",
            "     Species     n\n",
            "      <fctr> <int>\n",
            "1     setosa    50\n",
            "2 versicolor    50\n",
            "3  virginica    50\n"
          ]
        }
      ],
      "execution_count": 34,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "iris_count <- count(iris, Species, wt=Sepal.Length)\n",
        "print(iris_count)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 2\n",
            "     Species     n\n",
            "      <fctr> <dbl>\n",
            "1     setosa 250.3\n",
            "2 versicolor 296.8\n",
            "3  virginica 329.4\n"
          ]
        }
      ],
      "execution_count": 35,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x <- 1:9\n",
        "y <- c(x, 8, 9, 10)\n",
        "nth(x, 6)\n",
        "first(x)\n",
        "last(x)\n",
        "n_distinct(x)\n",
        "n_distinct(y)"
      ],
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "6"
            ],
            "text/markdown": [
              "6"
            ],
            "text/latex": [
              "6"
            ],
            "text/plain": [
              "[1] 6"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "1"
            ],
            "text/markdown": [
              "1"
            ],
            "text/latex": [
              "1"
            ],
            "text/plain": [
              "[1] 1"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "9"
            ],
            "text/markdown": [
              "9"
            ],
            "text/latex": [
              "9"
            ],
            "text/plain": [
              "[1] 9"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "9"
            ],
            "text/markdown": [
              "9"
            ],
            "text/latex": [
              "9"
            ],
            "text/plain": [
              "[1] 9"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "10"
            ],
            "text/markdown": [
              "10"
            ],
            "text/latex": [
              "10"
            ],
            "text/plain": [
              "[1] 10"
            ]
          },
          "metadata": {}
        }
      ],
      "execution_count": 36,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 数据分组\n",
        "\n",
        "```\n",
        "group_by(.data, ..., add = FALSE)\n",
        "```\n",
        "\ngroup_by()函数可以根据指定的数据列进行分组，分组完后可以在分组内部进行其他相关处理。例如：计算不同品种鸢尾花的萼片长度（Sepal_Length）均值，萼片宽度（Sepal_Width）均值，花瓣长度（Petal_Length）均值和花瓣宽度（Petal_Width）的均值："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris_grouped <- group_by(iris, Species)\n",
        "iris_grouped_summarised <- summarise_all(iris_grouped, funs(mean))\n",
        "print(iris_grouped_summarised)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 5\n",
            "     Species Sepal.Length Sepal.Width Petal.Length Petal.Width\n",
            "      <fctr>        <dbl>       <dbl>        <dbl>       <dbl>\n",
            "1     setosa        5.006       3.428        1.462       0.246\n",
            "2 versicolor        5.936       2.770        4.260       1.326\n",
            "3  virginica        6.588       2.974        5.552       2.026\n"
          ]
        }
      ],
      "execution_count": 37,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "dplyr提供了一种类似“管道”（Pipeline）功能的新语法，及操作符“`%>%`”。“`%>%`”可以将前面 的对象所谓第一个参数传递到后面的函数中去，例如：`x %>% f(y)`相当于`f(x, y)`。利用dplyr中的“`%>%`”操作符可以进一步改写成管道命令样式："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "iris %>%\n",
        "    group_by(Species) %>%\n",
        "    summarise_all(funs(mean)) %>%\n",
        "    print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 5\n",
            "     Species Sepal.Length Sepal.Width Petal.Length Petal.Width\n",
            "      <fctr>        <dbl>       <dbl>        <dbl>       <dbl>\n",
            "1     setosa        5.006       3.428        1.462       0.246\n",
            "2 versicolor        5.936       2.770        4.260       1.326\n",
            "3  virginica        6.588       2.974        5.552       2.026\n"
          ]
        }
      ],
      "execution_count": 38,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 关联\n",
        "\n",
        "```\n",
        "xxx_join(x, y, by = NULL, copy = FALSE, ...)\n",
        "```\n",
        "\n",
        "dplyr提供了6中数据关联的方法，xxx_join及其对应的含义如下所示：\n",
        "\n",
        "- `left_join` 左关联（保留x表数据，尽量关联y表数据）\n",
        "- `right_join` 右关联（保留y表数据，尽量关联x表数据）\n",
        "- `inner_join` 内关联（仅保留完全匹配项）\n",
        "- `full_join` 外关联（保留x表和y表中所有数据项）\n",
        "- `semi_join` 半关联（均在x表和y表中x表数据项）\n",
        "- `anti_join` 反关联（在x表中，但不在y表中x表数据项）"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "nationality <- tibble(\n",
        "    Name = c('Leo', 'Tom', 'John'),\n",
        "    Nationality = c('China', 'America', 'England')\n",
        ")\n",
        "gender <- tibble(\n",
        "    Name = c('Leo', 'Tom', 'Tracy'),\n",
        "    Gender = as.factor(c('Male', 'Male', 'Female'))\n",
        ")\n",
        "print(nationality)\n",
        "print(gender)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1   Leo       China\n",
            "2   Tom     America\n",
            "3  John     England\n",
            "# A tibble: 3 x 2\n",
            "   Name Gender\n",
            "  <chr> <fctr>\n",
            "1   Leo   Male\n",
            "2   Tom   Male\n",
            "3 Tracy Female\n"
          ]
        }
      ],
      "execution_count": 39,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "left_join(nationality, gender, by='Name') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 3\n",
            "   Name Nationality Gender\n",
            "  <chr>       <chr> <fctr>\n",
            "1   Leo       China   Male\n",
            "2   Tom     America   Male\n",
            "3  John     England   <NA>\n"
          ]
        }
      ],
      "execution_count": 40,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "right_join(nationality, gender, by='Name') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 3\n",
            "   Name Nationality Gender\n",
            "  <chr>       <chr> <fctr>\n",
            "1   Leo       China   Male\n",
            "2   Tom     America   Male\n",
            "3 Tracy        <NA> Female\n"
          ]
        }
      ],
      "execution_count": 41,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "inner_join(nationality, gender, by='Name') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 3\n",
            "   Name Nationality Gender\n",
            "  <chr>       <chr> <fctr>\n",
            "1   Leo       China   Male\n",
            "2   Tom     America   Male\n"
          ]
        }
      ],
      "execution_count": 42,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "full_join(nationality, gender, by='Name') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 4 x 3\n",
            "   Name Nationality Gender\n",
            "  <chr>       <chr> <fctr>\n",
            "1   Leo       China   Male\n",
            "2   Tom     America   Male\n",
            "3  John     England   <NA>\n",
            "4 Tracy        <NA> Female\n"
          ]
        }
      ],
      "execution_count": 43,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "semi_join(nationality, gender, by='Name') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1   Leo       China\n",
            "2   Tom     America\n"
          ]
        }
      ],
      "execution_count": 44,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "anti_join(nationality, gender, by='Name') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 1 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1  John     England\n"
          ]
        }
      ],
      "execution_count": 45,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 集合操作\n",
        "\n",
        "dplyr提供了3种集合操作函数，分别是交集，并集和差集：\n",
        "\n",
        "```\n",
        "intersect(x, y, ...)\n",
        "union(x, y, ...)\n",
        "setdiff(x, y, ...)\n",
        "```"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "nationality_1 <- tibble(\n",
        "    Name = c('Leo', 'Tom', 'John'),\n",
        "    Nationality = c('China', 'America', 'England')\n",
        ")\n",
        "nationality_2 <- tibble(\n",
        "    Name = c('Tom', 'John', 'Tracy'),\n",
        "    Nationality = c('America', 'England', 'America')\n",
        ")\n",
        "print(nationality_1)\n",
        "print(nationality_2)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1   Leo       China\n",
            "2   Tom     America\n",
            "3  John     England\n",
            "# A tibble: 3 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1   Tom     America\n",
            "2  John     England\n",
            "3 Tracy     America\n"
          ]
        }
      ],
      "execution_count": 46,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "intersect(nationality_1, nationality_2) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1   Tom     America\n",
            "2  John     England\n"
          ]
        }
      ],
      "execution_count": 47,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "union(nationality_1, nationality_2) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 4 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1 Tracy     America\n",
            "2  John     England\n",
            "3   Tom     America\n",
            "4   Leo       China\n"
          ]
        }
      ],
      "execution_count": 48,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "setdiff(nationality_1, nationality_2) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 1 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1   Leo       China\n"
          ]
        }
      ],
      "execution_count": 49,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 捆绑\n",
        "\ndplyr提供了数据集按行捆绑和按列捆绑的函数：`bind_rows()`和`bind_cols()`。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "bind_rows(nationality_1, nationality_2) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 6 x 2\n",
            "   Name Nationality\n",
            "  <chr>       <chr>\n",
            "1   Leo       China\n",
            "2   Tom     America\n",
            "3  John     England\n",
            "4   Tom     America\n",
            "5  John     England\n",
            "6 Tracy     America\n"
          ]
        }
      ],
      "execution_count": 50,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "bind_cols(nationality_1, nationality_2) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 4\n",
            "   Name Nationality Name1 Nationality1\n",
            "  <chr>       <chr> <chr>        <chr>\n",
            "1   Leo       China   Tom      America\n",
            "2   Tom     America  John      England\n",
            "3  John     England Tracy      America\n"
          ]
        }
      ],
      "execution_count": 51,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 数据重组\n",
        "\n",
        "数据的原始格式往往不一定适用后续的操作，例如：原始数据包含不同类型的多列数据，但在进行某些操作时（例如绘图），往往需要将多列数据汇集成行，有些时候则需要逆向的装换。tidyr提供了由列转行和由行转列的两个函数，分别为：\n",
        "\n",
        "```\n",
        "gather(data, key, value, ..., na.rm = FALSE, convert = FALSE)\n",
        "spread(data, key, value, fill = NA, convert = FALSE, drop = TRUE)\n",
        "```\n",
        "\n对于由列转行函数gather()，data表示待处理数据；key和value表示新生成数据集新的列名；...表示选取那些列进行转换，支持dplyr的select()语法；na.rm表示是否删除value列为NA的数据；convert表示是否对key列自动应用type.convert()函数。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "stocks <- tibble(\n",
        "    time = as.Date('2016-01-01') + 0:1,\n",
        "    x = rnorm(2, 0, 1),\n",
        "    y = rnorm(2, 0, 2),\n",
        "    z = rnorm(2, 0, 4))\n",
        "print(stocks)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Warning message in strptime(xx, f <- \"%Y-%m-%d\", tz = \"GMT\"):\n",
            "“unknown timezone 'default/Asia/Shanghai'”"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 4\n",
            "        time          x         y         z\n",
            "      <date>      <dbl>     <dbl>     <dbl>\n",
            "1 2016-01-01 -0.6344844 -3.274426 -1.654253\n",
            "2 2016-01-02  0.4306258 -3.143939  6.385287\n"
          ]
        }
      ],
      "execution_count": 52,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "stock_gathered <- gather(stocks, name, price, -time)\n",
        "print(stock_gathered)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 6 x 3\n",
            "        time  name      price\n",
            "      <date> <chr>      <dbl>\n",
            "1 2016-01-01     x -0.6344844\n",
            "2 2016-01-02     x  0.4306258\n",
            "3 2016-01-01     y -3.2744258\n",
            "4 2016-01-02     y -3.1439392\n",
            "5 2016-01-01     z -1.6542528\n",
            "6 2016-01-02     z  6.3852866\n"
          ]
        }
      ],
      "execution_count": 53,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "对于由行转列函数spread()函数，data表示待处理数据；key表示用于展开的列；value表示用于填充数据值的列；fill为当转换后无对应位置数据值填充的数据；convert表示是否对新生成的列自动应用type.convert()函数；drop如果为FALSE，将会保留数据中未出现的因子水平。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "spread(stock_gathered, name, price) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 4\n",
            "        time          x         y         z\n",
            "*     <date>      <dbl>     <dbl>     <dbl>\n",
            "1 2016-01-01 -0.6344844 -3.274426 -1.654253\n",
            "2 2016-01-02  0.4306258 -3.143939  6.385287\n"
          ]
        }
      ],
      "execution_count": 54,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "spread(stock_gathered, time, price) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 3\n",
            "   name `2016-01-01` `2016-01-02`\n",
            "* <chr>        <dbl>        <dbl>\n",
            "1     x   -0.6344844    0.4306258\n",
            "2     y   -3.2744258   -3.1439392\n",
            "3     z   -1.6542528    6.3852866\n"
          ]
        }
      ],
      "execution_count": 55,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "tidyr还提供了列分解和合并的函数，分别为：\n",
        "\n",
        "```\n",
        "separate(data, col, into, sep = \"[^[:alnum:]]+\", remove = TRUE, convert = FALSE, extra = \"error\", ...)\n",
        "unite(data, col, ..., sep = \"_\", remove = TRUE)\n",
        "```\n",
        "\n对于separate()函数，data表示待处理的数据；col为分解的列；into为分解后的列；sep为分解时使用的分隔符；remove表示是否从结果中删除原始的列；convert表示是否对新生成的列自动应用type.convert()函数；extra表示当分解后的个数与into的长度不同时的处理方式，如果为“error”则会报错，如果为“drop”则会补全不足的部分或删除多余的部分，如果为“merge”则至多会分割into长度的次数，即多余部分会补充到最后一个中。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "days <- tibble(\n",
        "    date = as.Date('2016-01-01') + 0:2\n",
        ")\n",
        "print(days)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 1\n",
            "        date\n",
            "      <date>\n",
            "1 2016-01-01\n",
            "2 2016-01-02\n",
            "3 2016-01-03\n"
          ]
        }
      ],
      "execution_count": 56,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "days_sep <- separate(days, date, c('year', 'month', 'day'))\n",
        "print(days_sep)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 3\n",
            "   year month   day\n",
            "* <chr> <chr> <chr>\n",
            "1  2016    01    01\n",
            "2  2016    01    02\n",
            "3  2016    01    03\n"
          ]
        }
      ],
      "execution_count": 57,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "kvs <- tibble(\n",
        "    kv = c(\"x:123\", \"y:error:7\")\n",
        ")\n",
        "print(kvs)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 1\n",
            "         kv\n",
            "      <chr>\n",
            "1     x:123\n",
            "2 y:error:7\n"
          ]
        }
      ],
      "execution_count": 58,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "kvs %>% separate(kv, c('key', 'value'), ':', extra='warn') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Warning message:\n",
            "“Too many values at 1 locations: 2”"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 2\n",
            "    key value\n",
            "* <chr> <chr>\n",
            "1     x   123\n",
            "2     y error\n"
          ]
        }
      ],
      "execution_count": 59,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "kvs %>% separate(kv, c('key', 'value'), ':', extra='drop') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 2\n",
            "    key value\n",
            "* <chr> <chr>\n",
            "1     x   123\n",
            "2     y error\n"
          ]
        }
      ],
      "execution_count": 60,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "kvs %>% separate(kv, c('key', 'value'), ':', extra='merge') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 2 x 2\n",
            "    key   value\n",
            "* <chr>   <chr>\n",
            "1     x     123\n",
            "2     y error:7\n"
          ]
        }
      ],
      "execution_count": 61,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "对于unite()函数，data表示待处理数据；col为合并后的列；...为待合并的列，支持dplyr的select()语法；sep为合并时的连接符；remove表示是否从结果中删除原始的列。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "days_sep %>% unite(date, year, month, day, sep='-') %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "# A tibble: 3 x 1\n",
            "        date\n",
            "*      <chr>\n",
            "1 2016-01-01\n",
            "2 2016-01-02\n",
            "3 2016-01-03\n"
          ]
        }
      ],
      "execution_count": 62,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 数据衍生\n",
        "\n",
        "数据衍生是指根据目前已有的数据列，按照一定的规则构造出新的数据列的过程。dplyr提供了两种用于数据衍生的函数mutate()和transmute()。\n",
        "\n",
        "```\n",
        "mutate(.data, ...)\n",
        "transmute(.data, ...)\n",
        "```\n",
        "\nmutate()函数会保留原始的数据列而transmute()函数不会，其中.data为待处理数据；...为衍生的新数据列。示例如下："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "mutate(iris, Sepal.Diff=Sepal.Length-Sepal.Width) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Length Sepal.Width Petal.Length Petal.Width    Species Sepal.Diff\n",
            "1            5.1         3.5          1.4         0.2     setosa        1.6\n",
            "2            4.9         3.0          1.4         0.2     setosa        1.9\n",
            "3            4.7         3.2          1.3         0.2     setosa        1.5\n",
            "4            4.6         3.1          1.5         0.2     setosa        1.5\n",
            "5            5.0         3.6          1.4         0.2     setosa        1.4\n",
            " [ reached getOption(\"max.print\") -- omitted 145 rows ]\n"
          ]
        }
      ],
      "execution_count": 63,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "transmute(iris, Sepal.Diff=Sepal.Length-Sepal.Width) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "    Sepal.Diff\n",
            "1          1.6\n",
            "2          1.9\n",
            "3          1.5\n",
            "4          1.5\n",
            "5          1.4\n",
            "6          1.5\n",
            "7          1.2\n",
            "8          1.6\n",
            "9          1.5\n",
            "10         1.8\n",
            "11         1.7\n",
            "12         1.4\n",
            "13         1.8\n",
            "14         1.3\n",
            "15         1.8\n",
            "16         1.3\n",
            "17         1.5\n",
            "18         1.6\n",
            "19         1.9\n",
            "20         1.3\n",
            "21         2.0\n",
            "22         1.4\n",
            "23         1.0\n",
            "24         1.8\n",
            "25         1.4\n",
            "26         2.0\n",
            "27         1.6\n",
            "28         1.7\n",
            "29         1.8\n",
            "30         1.5\n",
            " [ reached getOption(\"max.print\") -- omitted 120 rows ]\n"
          ]
        }
      ],
      "execution_count": 64,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## apply函数族及其扩展\n",
        "\n",
        "### apply函数族\n",
        "\n",
        "apply函数可以对矩阵或数组按照行或列进行运算，更一般而言，可以对多维数据的任意几个维度进行运算。apply函数的定义如下：\n",
        "\n",
        "```\n",
        "apply(X, MARGIN, FUN, ...)\n",
        "```\n",
        "\n其中X是一个矩阵或数组；MARGIN为进行运算的维度，1表示按行，2表示按列，c(1, 2)表示同时按行和按列，如果数据的各个维度有名称亦可以用名称；FUN为进行运算的函数；...为传递给FUN的可选参数。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "m <- matrix(1:12, 4, 3)\n",
        "print(m)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "     [,1] [,2] [,3]\n",
            "[1,]    1    5    9\n",
            "[2,]    2    6   10\n",
            "[3,]    3    7   11\n",
            "[4,]    4    8   12\n"
          ]
        }
      ],
      "execution_count": 65,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "apply(m, 1, mean) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1] 5 6 7 8\n"
          ]
        }
      ],
      "execution_count": 66,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "apply(m, 2, mean) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1]  2.5  6.5 10.5\n"
          ]
        }
      ],
      "execution_count": 67,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "lapply函数是对列表的每个元素进行运算，并返回运算后的结果。lapply函数定义如下：\n",
        "\n",
        "```\n",
        "lapply(X, FUN, ...)\n",
        "```"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "lst <- list(num=c(1, 1, 2, 3, 3, 4),\n",
        "           txt=c('a', 'a', 'b'),\n",
        "           logic=c(T, F, T, F))\n",
        "print(lst)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "$num\n",
            "[1] 1 1 2 3 3 4\n",
            "\n",
            "$txt\n",
            "[1] \"a\" \"a\" \"b\"\n",
            "\n",
            "$logic\n",
            "[1]  TRUE FALSE  TRUE FALSE\n",
            "\n"
          ]
        }
      ],
      "execution_count": 68,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lapply(lst, table) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "$num\n",
            "\n",
            "1 2 3 4 \n",
            "2 1 2 1 \n",
            "\n",
            "$txt\n",
            "\n",
            "a b \n",
            "2 1 \n",
            "\n",
            "$logic\n",
            "\n",
            "FALSE  TRUE \n",
            "    2     2 \n",
            "\n"
          ]
        }
      ],
      "execution_count": 69,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "sapply函数可以理解为一个更加友好版本的lapply，sapply也是对列表的每个元素进行运算，并返回运算后的结果。但根据返回结果的长度不同，sapply函数返回结果的类型也不同。如果对列表的每个元素计算返回的结果长度均为1，则sapply函数返回一个向量；如果对列表的每个元素计算返回的结果长度相同但大于1，则sapply函数返回一个矩阵；否则返回和lapply一样的列表。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "lst <- list(temperature=c(19, 20, 22, 18, 26, 30, 26, 23, 26, 21),\n",
        "           humidity=c(0.75, 0.80, 0.92, 0.62, 0.77, 0.56, 0.66, 0.67, 0.59, 0.78))\n",
        "print(lst)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "$temperature\n",
            " [1] 19 20 22 18 26 30 26 23 26 21\n",
            "\n",
            "$humidity\n",
            " [1] 0.75 0.80 0.92 0.62 0.77 0.56 0.66 0.67 0.59 0.78\n",
            "\n"
          ]
        }
      ],
      "execution_count": 70,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sapply(lst, mean) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "temperature    humidity \n",
            "     23.100       0.712 \n"
          ]
        }
      ],
      "execution_count": 71,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sapply(lst, quantile) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "     temperature humidity\n",
            "0%         18.00   0.5600\n",
            "25%        20.25   0.6300\n",
            "50%        22.50   0.7100\n",
            "75%        26.00   0.7775\n",
            "100%       30.00   0.9200\n"
          ]
        }
      ],
      "execution_count": 72,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "tapply函数主要用于分组统计，可以将一个函数应用到特定的分组组合。tapply函数定义如下：\n",
        "\n",
        "```\n",
        "tapply(X, INDEX, FUN = NULL, ..., simplify = TRUE)\n",
        "```\n",
        "\n其中X通常为一个向量；INDEX为一个list，list中的每个元素和X具有相同的长度；FUN为应用的函数；simplify表示是否简化结果，如果为TRUE，则返回一个数组，如果为FALSE，则返回一个列表。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "print(cars)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "   speed dist\n",
            "1      4    2\n",
            "2      4   10\n",
            "3      7    4\n",
            "4      7   22\n",
            "5      8   16\n",
            "6      9   10\n",
            "7     10   18\n",
            "8     10   26\n",
            "9     10   34\n",
            "10    11   17\n",
            "11    11   28\n",
            "12    12   14\n",
            "13    12   20\n",
            "14    12   24\n",
            "15    12   28\n",
            " [ reached getOption(\"max.print\") -- omitted 35 rows ]\n"
          ]
        }
      ],
      "execution_count": 73,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "code",
      "source": [
        "with(cars, tapply(dist, speed, mean)) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "       4        7        8        9       10       11       12       13       14       15       16 \n",
            " 6.00000 13.00000 16.00000 10.00000 26.00000 22.50000 21.50000 35.00000 50.50000 33.33333 36.00000 \n",
            "      17       18       19       20       22       23       24       25 \n",
            "40.66667 64.50000 50.00000 50.40000 66.00000 54.00000 93.75000 85.00000 \n"
          ]
        }
      ],
      "execution_count": 74,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "mapply函数是对一个函数，利用指定的参数集合进行计算。mapply函数的定义如下：\n",
        "\n",
        "```\n",
        "mapply(FUN, ..., MoreArgs = NULL, SIMPLIFY = TRUE, USE.NAMES = TRUE)\n",
        "```\n",
        "\n其中FUN为应用的函数；...为指定的参数集合；SIMPLIFY和tapply中的含义下同；USE.NAME为一个逻辑值参数，当...参数集合中的第一个参数具有名称或是一个字符向量时，则利用字符向量作为结果的名称。"
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "mapply(rep, 1:4, 1:4) %>% print"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[1]]\n",
            "[1] 1\n",
            "\n",
            "[[2]]\n",
            "[1] 2 2\n",
            "\n",
            "[[3]]\n",
            "[1] 3 3 3\n",
            "\n",
            "[[4]]\n",
            "[1] 4 4 4 4\n",
            "\n"
          ]
        }
      ],
      "execution_count": 75,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "上面的示例表示分别执行了rep(1, 1)，rep(2, 2)，rep(3, 3)和rep(4, 4)。\n",
        "\n",
        "## apply函数族扩展\n",
        "\n",
        "根据作者Hadley Wickham的描述，plyr包的主要思想为“分割-处理-合并”（Split-Apply-Combine）。我们通过一个示例来概括plyr扩展包的功能，首先我们比较循环版本和apply版本。plyr扩展包中的ozone数据集包含了美国中部地区不同地点和月份的臭氧数据，ozone数据集为一个24x24x72的矩阵，其三个维度分别为经度（long），维度（lat）和时间（time）。我们需要分析不同经纬度地区臭氧数据和时间变化的关系。\n",
        "\n先定义一些共用变量和函数："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "library(MASS)\n",
        "library(plyr)\n",
        "library(plotly)\n",
        "\n",
        "month <- ordered(rep(1:12, length = 72))\n",
        "\n",
        "deseas_fun <- function(value) {\n",
        "    rlm(value ~ month - 1, maxit=100) # Fit a linear model by robust regression using an M estimator.\n",
        "}"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n",
            "Attaching package: ‘MASS’\n",
            "\n",
            "The following object is masked from ‘package:dplyr’:\n",
            "\n",
            "    select\n",
            "\n",
            "--------------------------------------------------------------------------------------------------\n",
            "You have loaded plyr after dplyr - this is likely to cause problems.\n",
            "If you need functions from both plyr and dplyr, please load plyr first, then dplyr:\n",
            "library(plyr); library(dplyr)\n",
            "--------------------------------------------------------------------------------------------------\n",
            "\n",
            "Attaching package: ‘plyr’\n",
            "\n",
            "The following objects are masked from ‘package:dplyr’:\n",
            "\n",
            "    arrange, count, desc, failwith, id, mutate, rename, summarise, summarize\n",
            "\n",
            "Loading required package: ggplot2\n",
            "\n",
            "Attaching package: ‘plotly’\n",
            "\n",
            "The following object is masked from ‘package:ggplot2’:\n",
            "\n",
            "    last_plot\n",
            "\n",
            "The following objects are masked from ‘package:plyr’:\n",
            "\n",
            "    arrange, mutate, rename, summarise\n",
            "\n",
            "The following object is masked from ‘package:MASS’:\n",
            "\n",
            "    select\n",
            "\n",
            "The following object is masked from ‘package:stats’:\n",
            "\n",
            "    filter\n",
            "\n",
            "The following object is masked from ‘package:graphics’:\n",
            "\n",
            "    layout\n",
            "\n"
          ]
        }
      ],
      "execution_count": 76,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "循环版本的代码如下："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "models <- as.list(rep(NA, 24 * 24))\n",
        "dim(models) <- c(24, 24)\n",
        "\n",
        "deseas <- array(NA, c(24, 24, 72))\n",
        "dimnames(deseas) <- dimnames(ozone)\n",
        "\n",
        "for (i in seq_len(24)) {\n",
        "    for (j in seq_len(24)) {\n",
        "        model <- deseas_fun(ozone[i, j, ])\n",
        "        \n",
        "        models[[i, j]] <- model\n",
        "        deseas[i, j, ] <- resid(model)\n",
        "    }\n",
        "}"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Warning message in rlm.default(x, y, weights, method = method, wt.method = wt.method, :\n",
            "“'rlm' failed to converge in 100 steps”"
          ]
        }
      ],
      "execution_count": 77,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "apply版本的代码如下："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "models <- apply(ozone, 1:2, deseas_fun)\n",
        "resids_list <- lapply(models, resid)\n",
        "\n",
        "resids <- unlist(resids_list)\n",
        "dim(resids) <- c(72, 24, 24)\n",
        "deseas <- aperm(resids, c(2, 3, 1))\n",
        "dimnames(deseas) <- dimnames(ozone)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Warning message in rlm.default(x, y, weights, method = method, wt.method = wt.method, :\n",
            "“'rlm' failed to converge in 100 steps”"
          ]
        }
      ],
      "execution_count": 78,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "plyr版本代码如下："
      ],
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": [
        "models <- alply(ozone, 1:2, deseas_fun)\n",
        "deseas <- laply(models, resid)"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Warning message in rlm.default(x, y, weights, method = method, wt.method = wt.method, :\n",
            "“'rlm' failed to converge in 100 steps”"
          ]
        }
      ],
      "execution_count": 79,
      "metadata": {
        "collapsed": false,
        "outputHidden": false,
        "inputHidden": false
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "3种不同方法得到的结果一致，但从代码的简洁性上可以看出，利用plyr扩展包中的函数可以简洁高效的实现循环和系统apply函数族的功能。"
      ],
      "metadata": {}
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "ir",
      "language": "R",
      "display_name": "R"
    },
    "kernel_info": {
      "name": "ir"
    },
    "nteract": {
      "version": "0.2.0"
    },
    "language_info": {
      "name": "R",
      "codemirror_mode": "r",
      "pygments_lexer": "r",
      "mimetype": "text/x-r-source",
      "file_extension": ".r",
      "version": "3.4.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}